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Assistant Commissioner for Patents August 6 2001 

Washington, DC 20231 ' 

Sir: 

The following Preliminary Amendments and Remarks are 
respectfully submitted in connection with the above-identified 
application. 



IN THE SPECIFICATION ; 

Please amend the specification as follows: 

Before line 1, insert —This application is the national 
phase under 35 U.S.C. § 371 of PCT International Application No. 
PCT/JPOO/05861 which has an International filing date of 
August 30, 2000, which designated the United States of America 
and was not published in English. — 



AMENDMENTS 



Docket No. 1152-0278P 
Please replace the paragraph on page 6, lines 1-14, with the 
following rewritten paragraph: 

— The interpolation method disclosed in Japanese Patent 
Application laid-Open Hei 5 No. 30487 is an invention which adopts 
interpolation on a pixel row basis and provides improved 
processing speed compared to a typical interpolation on a pixel 
basis. However, in order to determine the edge of the pixel row, 
arithmetic operations between lines should be performed for the 
rows of at least three pixels to, more or less, eleven pixels 
while shifting pixels around the edge vicinity, hence the aspect 
of the long time requirement for arithmetic operations is still 
unsolved in this method. Further, this interpolation method is 
aimed at smoothing the outlines of images hence has the problem of 
inability to restore a pattern if the pattern has gaps at inter- 
lines as in an image depicted with nearly horizontal fine lines.— 

Please replace the paragraph on page 23, lines 7-12, with the 
following rewritten paragraph: 

— Interpolation implementing portion 140, is comprised of a 
pixel-row match searching portion 141, a search condition setup 
portion 142, a pre-interpolation pixel row setup portion 143, a 
matching pixel-row operating portion 144 and a correcting means 
145.— 

Please replace the paragraph on page 23 lines 22-25 
continuing on page 24, lines 1-2, with the following rewritten 
paragraph: 
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In this drawing. A, E, G and I designate input visual 
lines, and B, D, F, H designate lines to be interpolated. 
Designated at 21 (A33 to A35) , 22 (C27 to C31) , 23{E21 to E25) , 
24(G15 to G19) and 25(110 to 113), are black pattern portions 
depicting inclined fine line pattern 2, The other area represents 
white pattern portions. — 

Please replace the paragraph on page 2 6 lines 22-25 
continuing on page 27, lines 1-13, with the following rewritten 
paragraph: 

— When the virtual interpolation data is input form virtual 
interpolation database 120 to first matching pattern search 
portion 131 of interpolation segment determining processor 130, 
first matching pattern search portion 131 searches for matching 
patterns as to each of virtual interpolation line patterns 41 to 
44 and 51 to 54. For example, referring to a search for matching 
patterns of virtual interpolation line pattern 42, the segment of 
21, 22, 23, 24 and 25 on the virtual interpolation line D, based 
on the search range set up in search condition setup portion 133 
and based on the matching judgment conditions such as the sign, 
segment length, average differential luminance value etc., set in 
matching pattern condition setup portion 132, virtual 
interpolation line patterns matching virtual interpolation line 
pattern 42 are searched for from the virtual interpolation line 
patterns located below, to the lower left and to the right on the 
line F, as indicated by the arrows shown in Fig 5. — 
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Docket No. 1152-0278P 
Please replace the paragraph on page 33 lines 7-16, with the 
following rewritten paragraph: 

— In Fig. 10, visual pixel rows (black) are designated by 21 
to 25(A33 to A34, C27 to C31, E21 to E25, G15 to G19 and 110 to 
113); interpolation-designated segments (-) are designated by 41 to 
44(B27 to B31, D21 to D25, F15 to F19 and HlO to H13) , 
interpolation-designated segments (+) are designated by 51 to 
54(B33 to B34, D27 to d31, F21 to F25 and H15 to H19) ; and pre- 
interpolation pixel rows are designated by 61 to 64 and 71 to 
74(A30 to A34, C24, to C28, C30, to C34, E18 to E22, E24, to E28, 
G12 to G16, G18 to G22 and 112 to 116).— 

Please replace the paragraph on page 33 lines 17-21, with the 
following rewritten paragraph: 

— In Fig. 11, visual pixel rows (black) are designated by 21 
to 25(A33 to A34, C27 to C31, E21 to E25, G15 to G19 and 110 to 
113); and interpolation pixel rows are designated by 81 to 84 (B27 
to B31, B33 to B34, D21 to D25, D27 to D31, F15 to F19, F21 to F25 
HlO to H13 and HIS to H19).— 

Please replace the paragraph on page 33 lines 22-25 
continuing on page 34, lines 1-4 with the following rewritten 
paragraph: 

—First, referring to an example of setting up a pixel row 
for interpolation in interpolation-designated segment 42 (D21 to 
D25), pre-interpolation pixel row setup portion 143, based on the 
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segment starting positions and length data of interpolation- 
designated segments 42 and 43(D21 to D25, F15 to F19) obtained 
from virtual interpolation database 120, sets up pre-interpolation 
pixel rows 62 and 63 (C24 to C28, E18 to E22), on lines C and E. — 

Please replace the paragraph on page 34 lines 16-22, with the 
following rewritten paragraph: 

—Then, the average of pixel data of C24 and ElB of pre- 
interpolation pixel rows 62 and 63(C24 to C28 and ElB to E22) is 
calculated to produce the pixel data of D21 of interpolation pixel 
row 82(D21 to D25) . Thus, average of pixel data is calculated 
sequentially up to C28 and E22 so as to produce the pixel data of 
interpolation pixel row 82(D21 to D25) . — 
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REMARKS 

The specification has been amended to provide a cross- 
reference to the previously filed International Application. The 
specification has also been amended to correct typographical 
errors. Entry of the above amendments is earnestly solicited. An 
early and favorable first action on the merits is earnestly 
solicited. 

Attached hereto is a marked-up version of the changes made to 
the application by this Preliminary Amendment. 

If necessary, the Commissioner is hereby authorized in this, 
concurrent, and future replies, to charge payment or credit any 
overpayment to Deposit Account No. 02-2448 for any additional 
fees required under 37 C.F.R. § 1.16 or under 37 C.F.R. § 1.17; 
particularly, extension of time fees. 



Respectfully submitted, 

BIRCH, STEWART, KOLASCH & BIRCH, LLP 




P.O. Box 747 

DJD/rem Falls Church, VA 22040-0747 

1152-0278P (703) 205-8000 
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The specification has been amended to provide cross- 
referencing to the International Application. 

The paragraph on page 6, lines 1-14, has been amended as 
follows : 

The interpolation method disclosed in Japanese Patent 
Application laid-Open Hei 5 No. 30487 is an invention which adopts 
interpolation on a pixel row basis and provides improved 
processing speed compared to a typical interpolation on a pixel 
basis. However, in order to determine the edge of the pixel row, 
arithmetic operations between lines should be performed for the 
rows of at least three pixels to, more or less, eleven pixels 
while shifting pixels around the edge vicinity, hence the aspect 
of the long time requirement for arithmetic operations is still 
unsolved in this method. Further, this interpolation method is 
aimed at smoothing the outlines of [pixels] images hence has the 
problem of inability to restore a pattern if the pattern has gaps 
at inter-lines as in an image depicted with nearly horizontal fine 
lines . 

The paragraph on page 23, lines 7-12, has been amended as 
follows : 

Interpolation implementing portion 14 0, is comprised of a 
pixel-row match searching portion 141, a search condition setup 
portion 142, [a pre-interpolation pixel row setup portion 142,] a 
pre-interpolation pixel row setup portion 143, a matching pixel- 
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row operating portion 144 and a correcting means 145. 

The paragraph on page 23 lines 22-25 continuing on page 24, 
lines 1-2, has been amended as follows: 

In this drawing. A, C, E, G and I designate input visual 
lines, and B, D, F, H designate lines to be interpolated. 
Designated at 21(A33 to A35) , 22 (C27 to C31) , 23(E21 to E25) , 
24(G15 to G19) and 25([ilO to il3] ) ( IIP to 113 ) , are black pattern 
portions depicting inclined fine line pattern 2. The other area 
represents white pattern portions. 

The paragraph on page 26 lines 22-25 continuing on page 27, 
lines 1-13, has been amended as follows: 

When the virtual interpolation data is input form virtual 
interpolation database 120 to first matching pattern search 
portion 131 of interpolation segment determining processor 130, 
first matching pattern search portion 131 searches for matching 
patterns as to each of virtual interpolation line patterns 41 to 
44 and 51 to 54. For example, referring to a search for matching 
patterns of virtual interpolation line pattern 42, the segment of 
21, 22, [23 and 24] 23, 24 and 25 on the virtual interpolation line 
D, based on the search range set up in search condition setup 
portion 133 and based on the matching judgment conditions such as 
the sign, segment length, average differential luminance value 
etc., set in matching pattern condition setup portion 132, virtual 
interpolation line patterns matching virtual interpolation line 
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pattern 42 are searched for from the virtual interpolation line 
patterns located below, to the lower left and to the right on the 
line F, as indicated by the arrows shown in Fig 5. 

The paragraph on page 33 lines 7-16, has been amended as 
follows : 

In Fig. 10, visual pixel rows (black) are designated by 21 to 
25(A33 to A34, C27 to C31, E21 to E25, G15 to G19 and 110 to 113); 
interpolation-designated segments (-) are designated by 41 to 
44(B27 to B31, D21 to D25, F15 to F19 and HID to H13) , 
interpolation-designated segments (+) are designated by 51 to 
54(B33 to B34, D27 to D31, F21 to F25 and H15 to [19] H19) ; and 
pre-interpolation pixel rows are designated by 61 to 64 and 71 to 
74(A30 to A34, C24, to C28, C30, to C34, E18 to E22, E24, to E28, 
G12 to G16, G18 to G22 and 112 to 116) . 

The paragraph on page 33 lines 17-21, has been amended as 
follows : 

In Fig. 11, visual pixel rows (black) are designated by 21 to 
25(A33 to A34, C27 to C31, E21 to E25, G15 to G19 and 110 to 113); 
and interpolation pixel rows are designated by 81 to 84(B27 to 
B31, B33 to B34, D21 to D25, D27 to D31, F15 to F19, F21 to [F15] 
F25 HIO to H13 and H15 to H19) . 

The paragraph on page 33 lines 22-25 continuing on page 34, 
lines 1-4 has been amended as follows: 

9 



Docket No. 1152-0278P 
First, referring to an example of setting up a pixel row for 
interpolation in interpolation-designated segment [42(D-21 to 25)] 
42 (D21 to D25 ) , pre-interpolation pixel row setup portion 143, 
based on the segment starting positions and length data of 
interpolation-designated segments 42 and 43([D-21] D21 to D25, [F- 
15] F15 to F19) obtained from virtual interpolation database 120, 
sets up pre-interpolation pixel rows 62 and 63{C24 to C28, E18 to 
E22) , on lines C and E. 

The paragraph on page 34 lines 16-22, has been amended as 
follows : 

Then, the average of pixel data of C24 and E18 of pre- 
interpolation pixel rows 62 and 63(C24 to [C26] C28 and E18 to 
E22) is calculated to produce the pixel data of D21 of 
interpolation pixel row 82(D21 to D25) , Thus, average of pixel 
data is calculated sequentially up to C28 and E22 so as to produce 
the pixel data of interpolation pixel row 82 (D21 to D25) . 
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DESCRIPTION 



IMAGE INTEROPERATION SYSTEM AND IMAGE INTERPOLATION METHOD 

5 Technical Field 

The present invention relates to an image interpolation 
system and image interpolation method, and in particular 
relates to an image interpolation system and image 
interpolation process by line interpolation for upconverting 
10 a certain television system into another television system 
with a different number of scanning lines or for creating 
a frame of image from a field of image when an interlaced 
scanning video signal is converted into a sequential scanning 
video signal. 



Background Art 

Conventionally, for scanning line interpolation which 
creates a progressive video image from an interlaced video 
image in order to upcovert a television system into another 

20 with a different number of scanning lines or in order to render 
an image at a higher definition, intraframe interpolation 
is performed for still images. For motion pictures, 
intra- field interpolation is performed because the 
correlation within a frame may corrupt in a motion picture 

25 and defective conversion such as jaggedness may occur when 
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moving vertical lines are interpolated with the frame. 

As simple methods of intra-field interpolation, lines 
to be interpolated may be processed by line interpolation 
with the lines directly above or may be processed by line 
5 interpolation using the average of the lines above and below. 
The former interpolation has been accompanied by jaggedness 
in image outlines for patterns such as inclined lines etc., 
having less correlation with respect to the vertical direction , 
The latter interpolation method has been accompanied by image 

10 degradation such as unclearness etc., in the image. 

As a technique for solving these drawbacks, the 
interpolation method of Japanese Patent Application Laid-Open 
Sho 63 No. 187785 discloses interpolation whereby every pixel 
on lines to be interpolated is interpolated according to the 

15 information of the pixels which reside around the pixel and 
in a direction in which the pixel to be interpolated has the 
strongest correlation . First , in order to obtain information 
as to the radial direction in which the pixel to be interpolated 
has the strongest correlation of pixel data, absolute values 

20 of the difference between neighboring pixels with respect 
to the vertical direction and right and left diagonal 
directions are checked so as to determine that the direction 
along which the absolute differential value becomes minimum 
is the direction presenting the strongest correlation , whereby 

25 the average value of the pixels in that direction is determined 
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and allotted, to the value of the pixel to be interpolated. 

However, this method needs to calculate the absolute 
differential values at least in the three directions as above 
and determine the minimum value of these absolute differential 
5 values to determine the value of the pixel to be interpolated 
for each of all the pixels constituting the lines to be 
interpolated. Therefore, it takes a long time for 
interpolation processing. Further, since the above sequence 
of processing is also performed for all the area including 

10 pixels other than at the edges in the image (where the pixel 
values vary little) or for the cases where there is little 
difference in the degree of correlation depending on the 
directions, the time for processing is wastefully consumed, 
hence the speed of interpolation is low so that it is also 

15 difficult to enlarge the range of pixels of which the 
correlation is checked. 

Therefore , when interpolation is implemented for an image 
including an inclined line having a small inclination where 
pixels located considerably apart from each other have 

20 correlation, it is impossible to perform interpolation based 
on the strong correlation between these pixels, sill leading 
to production of Jaggedness along the edges of the inclined 
portion with poor interpolation precision. 

In order to overcome the drawback of the above publication , 

25 JapanesePatent Application Laid-open Hei 5 No. 30487 discloses 



a method which Improves the processing speed of interpolation 
and improves the interpolation accuracy by enlarging the range 
of searching for correlation with a corresponding enhancement 
of the processing speed. 
5 Next, the image interpolation method disclosed in this 

publication will be described with reference to Figs . 14 to 
17. 

To begin with, in this interpolation method, as shown 
in Fig. 14, the pixels on neighboring lines, namely n-th line 

10 and (n+l)-th line, on a two-dimensional image are compared 
so as to detect edge points a and b on respective lines in 
the two-dimensional image . It is assumed that the pixels which 
are on the line to be interpolated, other than the points 
between the edge points a and b, may be interpolated by the 

15 pixels on either of the adjacent lines so as to determine 
the edge on the line to be interpolated. 

Next, as shown in Fig. 15, around the edges a and b, a 
neighboring pixel row (3,0) , which is centered at an observed 
pixel (pixel A in this case) on either of the neighboring 

20 lines (n-th line) and is defined by 'the number of pixels' 
and ' the amount of shift from the position to be interpolated' 
is created while an associated pixel row (3,1) correlated 
with the neighboring pixel row is selected from the other 
adjacent line(n+l)-th line. 

25 Here, as a general form for establishing correlation 
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between the neighboring pixel row and the associated pixel 
row, the neighboring pixel row is defined to be (2m+l,0) and 
the associated pixel row is defined to be (2ni+l, +Y) where 
m and Y are sequentially changed as m=Y=l, 2, 3, 4, 5. 
5 Next , as shown in Fig. 16 , based on the operations between 

each pair of pixels indicated by the arrow in the drawing, 
in neighboring pixel row( 3,0) and associated pixel row{ 3,1), 
whether the level difference between the pixels falls within 
the predetermined range is checked so as to determine the 

10 correlation . In the case shown in the drawing , no correlation 
exists between the centered pair of pixels. Hence, m and Y 
are set equal to 2 , the operations between each pair of pixels , 
indicated by the arrows in the drawing are performed for the 
neighboring pixel row (5,0) and associated pixel row (5,2) . 

15 When the presence of correlation is recognized from the pairs 
of pixels indicated by the arrows in the drawing, the amount 
of shift from the position to be interpolated is known to 
be two pixels. That is, as shown in Fig. 17, point a and point 
b are two pixels shifted from one another and hence 

20 interpolation is implemented using a pixel row, which is 
created by shifting the selected pixel row or the neighboring 
pixel row by half the determined number of pixels in the 
direction opposite to the shifted direction, specifically, 
the pixel row centered by point c, which is one pixel shifted 

25 rightward from pixel A. 
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The interpolation method disclosed in Japanese Patent 
Application Laid-Open Hei 5 No. 30487 is an invention which 
adopts interpolation on a pixel row basis and provides improved 
processing speed compared to a typical interpolation on a 
5 pixel basis. However, in order to determine the edge of the 
pixel row, arithmetic operations between lines should be 
performed for the rows of at least three pixels to, more or 
less, eleven pixels while shifting pixels around the edge 
vicinity, hence the aspect of the long time requirement for 

10 arithmetic operations is still unsolved in this method. 

Further , this interpolation method is aimed at smoothing the 
outlines of pixels, hence has the problem of inability to 
restore a pattern if the pattern has gaps at inter- lines as 
in an image depicted with nearly horizontal fine lines . 

15 In view of the above problems of the conventional art, 

the object of the present invention is to provide an image 
interpolation system and image interpolation method which 
can realize real-time interpolation of various video pictures 
including patterns with fine lines, edges etc., by sharply 

20 reducing the processing speed. 

Disclosure of Invention 

The present invention has been completed in order to 
attain the above object and is configured as f ollows : - 
25 First, the first aspect of the present invention resides 
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in an image interpolation system for interpolating the gaps 
between the lines forming an image, comprising: 

a virtual interpolation data generating means for 
generating virtual interpolation data of inter- lines between 
5 the lines of the input image, based on the input image line 
data; and 

an interpolating means for Interpolating the pixels 
between input image lines, based on the generated virtual 
interpolation data. 

10 Next . the second aspect of the present invention resides 

in an image interpolation system for interpolating the gaps 
between the lines forming an image, comprising: 

a virtual interpolation data generating means for 
generating virtual interpolation data of inter- lines between 

15 the lines of the input image, based on the input image line 
data; and 

an interpolating means which, based on the generated 
virtual interpolation data, generates pre-interpolation 
pixels on the input image lines , and interpolates the pixels 
20 between input image lines above and below by performing mutual 
operations between pre-interpolation pixels generated on the 
input image lines above and below. 

Further, the third aspect of the present invention 
resides in an image interpolation system for interpolating 
25 the gaps between the lines forming an image, comprising: 



a virtual interpolation data generating means for 
generating virtual interpolation data of inter -lines between 
the lines of the input image, based on the input image line 
data; 

an Interpolation segment determining means for 
determining segments to be interpolated between the input 
image lines and the direction of interpolation, based on the 
generated virtual interpolation data; and 

an interpolating means which generates 
pre-interpolation pixels on the input image lines, based on 
the generated virtual interpolation data and the determined 
data of the segments to be interpolated, determined by the 
interpolation segment determining means, and interpolates 
the pixels between input image lines based on the generated 
pre-interpolation pixels. 

Next, the fourth aspect of the present invention resides 
in the image interpolation system having the above third 
feature and characterized in that the interpolation segment 
determining means comprises: a search condition setup means 
for setting up a pattern search range; a matching pattern 
condition setup means for setting up matching pattern 
conditions; and a first matching pattern searching means for 
searching for matching patterns based on the conditions 
designated by the search condition setup means and by the 
matching pattern condition setup means. 
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Further, the fifth aspect of the present invention 
resides in the image interpolation system having the above 
third feature and characterized in that the interpolation 
segment determining means comprises : a search condition setup 
5 means for setting up a pattern search range ; a matching pattern 
condition setup means for setting up matching pattern 
conditions; a first matching pattern searching means for 
searching matching patterns based on the conditions des ignat ed 
by the search condition setup means and by the matching pattern 

10 condition setup means; a directional vector extracting means 
for extracting the direction of the vector of the detected 
matching patterns; and a second matching pattern searching 
means for searching for matching patterns existing in the 
extracted direction of the vector, based on the conditions 

15 designated by the search condition setup means and by the 
matching pattern condition setup means. 

Next , the sixth aspect of the present invention resides 
in the image interpolation system having the above first 
feature and characterized in that the virtual interpolation 

20 data generated by the virtual interpolation data generating 
means is constructed of units of pixel-rows of data. 

Further, the seventh aspect of the present invention 
resides in the image interpolation system having the above 
second feature and characterized in that the virtual 

25 interpolation data generated by the virtual interpolation 
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data generating means is constructed of units of pixel-rows 
of data. 

Next, the eighth aspect of the present invention resides 
in the image interpolation system having the above third 
5 feature and characterized in that the virtual interpolation 
data generated by the virtual interpolation data generating 
means is constructed of units of pixel-rows of data. 

Also, the ninth aspect of the present invention resides 
In the Image interpolation system having the above fourth 
10 feature and characterized in that the virtual interpolation 
data generated by the virtual interpolation data generating 
means is constructed of units of pixel-rows of data. 

Next, the tenth aspect of the present invention resides 
in the image interpolation system having the above fifth 
15 feature and characterized in that the virtual interpolation 
data generated by the virtual interpolation data generating 
means is constructed of units of pixel -rows of data. 

Further, the eleventh aspect of the present invention 
resides in the image interpolation system having the above 
20 first feature and characterized in that the virtual 

interpolation data generating means comprises : an inter-pixel 
operating means for calculating the difference in pixel data 
between the pixels on the neighboring input image lines; and 
a normalizing means for classifying the pixels into multiple 
25 classes according to the calculated value of the difference 



in pixel data; and a pattern extracting means for extracting 
rows of pixels normalized and classified in an identical class 
as patterns . 

Next , the twelfth aspect of the present invention resides 
in the image interpolation system having the above second 
feature and characterized in that the virtual interpolation 
data generating means comprises: an inter-pixel operating 
means for calculating the difference in pixel data between 
the pixels on the neighboring input image lines; and a 
normalizing means for classifying the pixels into multiple 
classes according to the calculated value of the difference 
in pixel data; and a pattern extracting means for extracting 
rows of pixels normalized and classified in an identical class 
as patterns . 

Also, the thirteenth aspect of the present invention 
resides in the image interpolation system having the above 
third feature and characterized in that the virtual 
interpolation data generating means comprises : an inter-pixel 
operating means for calculating the difference in pixel data 
between the pixels on the neighboring input image lines; and 
a normalizing means for classifying the pixels into multiple 
classes according to the calculated value of the difference 
in pixel data; and a pattern extracting means for extracting 
rows of pixels normalized and classified in an identical class 
as patterns . 
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Next, the fourteenth aspect of the present invention 
resides in the image interpolation system having the above 
fourth feature and characterized in that the virtual 
interpolation data generating means comprises : an inter-pixel 
operating means for calculating the difference in pixel data 
between the pixels on the neighboring input image lines; and 
a normalizing means for classifying the pixels into multiple 
classes according to the calculated value of the difference 
in pixel data; and a pattern extracting means for extracting 
rows of pixels normalized and classified in an identical class 
as patterns. 

Further, the fifteenth aspect of the present invention 
resides in the image interpolation system having the above 
fifth feature and characterized in that the virtual 
interpolation data generating means comprises : an inter-pixel 
operating means for calculating the difference in pixel data 
between the pixels on the neighboring input image lines; and 
a normalizing means for classifying the pixels into multiple 
classes according to the calculated value of the difference 
in pixel data; and a pattern extracting means for extracting 
rows of pixels normalized and classified in an identical class 
as patterns. 

Next, the sixteenth aspect of the present invention 
resides in the image interpolation system having the above 
sixth through tenth features and characterized in that 
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according to Claims 6 through 10, wherein the virtual 
interpolation data generating means comprises : an inter-pixel 
operating means for calculating the difference in pixel data 
between the pixels on the neighboring input image lines ; and 
a normalizing means for classifying the pixels into multiple 
classes according to the calculated value of the difference 
in pixel data; and a pattern extracting means for extracting 
rows of pixels normalized and classified in an identical class 
as patterns . 

Then, the seventeenth aspect of the present invention 
resides in the image interpolation system having the above 
first feature and characterized in that the virtual 
interpolation data generating means comprises : an inter-pixel 
operating means for calculating the difference in pixel data 
between the pixels on the neighboring Input image lines; a 
normalizing means for classifying the pixels into multiple 
classes according to the calculated value of the difference 
in pixel data; a pattern extracting means for extracting rows 
of pixels normalized and classified in an identical class 
as patterns; and a coring means for judging the patterns 
extracted on the same line to be interpolated based on the 
predetermined threshold and editing them. 

Next, the eighteenth aspect of the present invention 
resides in the image interpolation system having the above 
second feature and characterized in that the virtual 
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interpolation data generating means comprises : an inter-pixel 
operating means for calculating the difference in pixel data 
between the pixels on the neighboring input image lines; a 
normalizing means for classifying the pixels into multiple 
classes according to the calculated value of the difference 
in pixel data; a pattern extracting means for extracting rows 
of pixels normalized and classified in an identical class 
as patterns; and a coring means for judging the patterns 
extracted on the same line to be interpolated based on the 
predetermined threshold and editing them. 

Further, the nineteenth aspect of the present invention 
resides in the image interpolation system having the above 
third feature and characterized in that the virtual 
interpolation data generating means comprises : an inter-pixel 
operating means for calculating the difference in pixel data 
between the pixels on the neighboring input image lines; a 
normalizing means for classifying the pixels into multiple 
classes according to the calculated value of the difference 
in pixel data; a pattern extracting means for extracting rows 
of pixels normalized and classified in an identical class 
as patterns; and a coring means for Judging the patterns 
extracted on the same line to be interpolated based on the 
predetermined threshold and editing them. 

Next, the twentieth aspect of the present invention 
resides in the image interpolation system having the above 
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fourth feature and characterized In that the virtual 
interpolation data generating means comprises : an inter-pixel 
operating means for calculating the difference in pixel data 
between the pixels on the neighboring input image lines; a 
5 normalizing means for classifying the pixels into multiple 
classes according to the calculated value of the difference 
in pixel data; a pattern extracting means for extracting rows 
of pixels normalized and classified in an identical class 
as patterns; and a coring means for Judging the patterns 

10. extracted on the same line to be interpolated based on the 
predetermined threshold and editing them. 

Further, the twenty-first aspect of the present invention 
resides in the image interpolation system having the above 
fifth feature and characterized in that the virtual 

15 interpolation data generating means comprises : an inter-pixel 
operating means for calculating the difference in pixel data 
between the pixels on the neighboring input image lines; a 
normalizing means for classifying the pixels into multiple 
classes according to the calculated value of the difference 

20 in pixel data; a pattern extracting means for extracting rows 
of pixels normalized and classified in an identical class 
as patterns; and a coring means for judging the patterns 
extracted on the same line to be interpolated based on the 
predetermined threshold and editing them. 

25 Next, the twenty- second aspect of the present invention 



16 



resides in the image interpolation system having the above 
sixteenth feature and characterized in that the virtual 
interpolation data generating means comprises : an inter-pixel 
operating means for calculating the difference in pixel data 
between the pixels on the neighboring input image lines; a 
normalizing means for classifying the pixels into multiple 
classes according to the calculated value of the difference 
in pixel data; a pattern extracting means for extracting rows 
of pixels normalized and classified in an identical class 
as patterns; and a coring means for judging the patterns 
extracted on the same line to be interpolated based on the 
predetermined threshold and editing them. 

Furthermore, the twenty- third aspect of the present 
invention resides in the image interpolation system having 
the above seventeenth feature and characterized in that the 
coringmeans comprises : an intra-pat tern pixel operatingmeans 
for calculating the average of the pixel differential values 
of individual pixels in each pattern; and a pattern editing 
means for judging the average value based on the predetermined 
threshold and editing patterns. 

Next, the twenty-fourth aspect of the present invention 
resides in the image interpolation system having the above 
eighteenth feature and characterized in that the coring means 
comprises: an intra-pattern pixel operating means for 
calculating the average of the pixel differential values of 
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individual pixels in each pattern; and a pattern editing means 
for judging the average value based on the predetermined 
threshold and editing patterns - 

Further , the twenty- fifth aspect of the present invention 
5 resides in the image interpolation system having the above 
nineteenth feature and characterized in that the coring means 
comprises: an intra-pattern pixel operating means for 
calculating the average of the pixel differential values of 
individual pixels in each pattern; and a pattern editing means 

10 for judging the average value based on the predetermined 
threshold and editing patterns. 

Next, the twenty-sixth aspect of the present invention 
resides in the image interpolation system having the above 
twentieth feature and characterized in that the coring means 

15 comprises: an intra-pattern pixel operating means for 

calculating the average of the pixel differential values of 
individual pixels in each pattern; and a pattern editing means 
for judging the average value based on the predeteirmined 
threshold and editing patterns. 

20 Further, the twenty- seventh aspect of the present 

invention resides in the image interpolation system having 
the above twenty-first feature and characterized in that the 
coringmeans comprises : an intra-pattern pixel operatingmeans 
for calculating the average of the pixel differential values 

25 of individual pixels in each pattern; and a pattern editing 



18 



means for judging the average value based on the predetermined 
threshold and editing patterns. 

Next, the twenty-eighth aspect of the present invention 
resides in the image interpolation system having the above 
twenty- second feature and characterized in that the coring 
means comprises: an intra-pattern pixel operating means for 
calculating the average of the pixel differential values of 
individual pixels in each pattern; and a pattern editing means 
for judging the average value based on the predetermined 
threshold and editing patterns. 

Moreover, the twenty-nine aspect of the present invention 
resides in an image interpolation method for interpolating 
the gaps between the lines forming an image, comprising: 
a step of generating virtual interpolation data of 
inter- lines between the lines of the input image, based on 
the input image line data; and 

a step of interpolating the pixels between input image 
lines, based on the generated virtual interpolation data. 

Next, the thirtieth aspect of the present invention 
resides in an image interpolation method for interpolating 
the gaps between the lines forming an image, comprising: 
a step of generating virtual interpolation data of 
inter-lines between the lines of the input image, based on 
the input image line data; and 

a step of generating pre- interpolation pixels on the 
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input image lines based on the generated virtual interpolation 
data and interpolating the pixels between input image lines 
above and below by performing mutual operations between 
pre -interpolation pixels generated on the input image lines 
above and below. 

Finally, the thirty-first aspect of the present invention 
resides in an image interpolation method for interpolating 
the gaps between the lines forming an image, comprising: 
a step of generating virtual interpolation data of 
inter- lines between the lines of the input image, based on 
the input image line data; 

a step of determining segments to be interpolated between 
the input image lines and/or the direction of interpolation, 
based on the generated virtual interpolation data; and 

a step of generating pre -interpolation pixels on the 
input image lines . based on the generated virtual 
interpolation data and the determined data of the segments 
to be interpolated, and interpolating the pixels between input 
image lines based on the generated pre -interpolation pixels. 

Brief Description of the Drawings 

Fig.l is a block diagram showing a configuration of an 
image interpolation system in accordance with the first 
embodiment of the present invention. 

Fig - 2 is a view showing a monitor screen on which a frame 
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of Visual image having a black fine line pattern on a white 
background is displayed as an example of an input video picture 
for explaining the first embodiment. 

Fig. 3 is a chart showing input visual lines and lines 
to be interpolated at an enlargement point 3 shown in Fig. 2. 

Fig. 4 is a chart showing virtual interpolation line 
patterns generated by a virtual interpolation line pattern 
generator 100 shown in Fig.l. 

Fig. 5 is a chart for illustrating how the first matching 
pattern searching portion 131 and matching pattern locating 
portion 134 shown in Fig.l search for matching patterns. 

Fig. 6 is a chart for illustrating how the directional 
vector extracting portion 135 and second matching pattern 
searching portion 136 shown in Fig.l search for a vector. 

Fig. 7 is a chart showing pre-interpolation pixel rows 
(= match pixel rows) 61 to 63 and 71 to 73 set up by the 
pre-interpolation pixel row setting portion 143 shown in 
Fig.l. 

Fig. 8 is a chart showing pixel rows 81 to 84 which are 
created by interpolation of the interpolation implementing 
portion 140 shown in Fig.l. 

Fig. 9 is a block diagram showing a configuration of an 
image interpolation system in accordance with the second 
embodiment of the present invention. 

Fig. 10 is a chart for illustrating the positional 



21 



relationship between visual lines and pixel rows on the lines 
to be interpolated, for explaining the second embodiment. 

Fig. 11 is a chart for illustrating the positional 
relationship between visual lines and pixel rows on the lines 
5 to be interpolated, for explaining the second embodiment. 

Fig. 12 is a block diagram showing a configuration of 
an image interpolation system in accordance with the third 
embodiment - 

Fig. 13 is a chart for illustrating the positional 
10 relationship between visual lines and pixel rows on the lines 
to be interpolated, for explaining the third embodiment. 

Fig. 14 is a diagram for illustrating the positional 
relationship between visual lines and pixel rows on the lines 
to be interpolated, in an image interpolation method in 
15 accordance with a prior art technique. 

Fig. 15 a diagram for illustrating a process of 
determining the correlation between a neighboring pixel row 
and an associated pixel row in an image interpolation method 
in accordance with a prior art technique. 
20 Fig. 16 a diagram for illustrating another process of 

determining the correlation between a neighboring pixel row 
and an associated pixel row in an image interpolation method 
in accordance with a prior art technique. 

Fig. 17 is a diagram for illustrating interpolation at 
25 an edge in an image interpolation method in accordance with 
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a prior art technique. 

Best Mode for Carrying Out the Invention 

The first embodiment of the present invention will be 
5 explained with reference to Figs .1 to 8 . 

Fig.l is a block diagram showing a configuration of an 
image interpolation system in accordance with the present 
embodiment . 

In this drawing, 100 designates a virtual interpolation 
10 line pattern generator which receives an input video picture 
and generates virtual interpolation line patterns, 120 
designates a virtual interpolation database for storing the 
generated virtual line patterns, 130 designates an 
interpolating segment detennining portion for determining 
15 interpolating segments, and the direction of interpolation, 
and 140 designates an interpolation implementing portion for 
performing interpolation based on arithmetically operated 
data. 

Virtual interpolation line pattern generator 100 is 
20 comprised of an inter-pixel differential luminance operating 
portion 101, a normalization processor 102, a pattern 
extracting portion 103, a coring processor 110, an 
intra-pattern differential luminance operating portion 111, 
a pattern edit judging portion 112 and a pattern edit 
25 implementing portion 113. 
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Interpolating segment determining portion 130 is 
comprised of a first matching pattern searching portion 131, 
a matching pattern condition setup portion 132, a search 
condition setup portion 133, a matching pattern locating 
portion 134, a directional vector extracting portion 135 and 
a second matching pattern searching portion 136. 

Interpolation implementing portion 140 is comprised of 
a pixel-row match searching portion 141, a search condition 
setup portion 142 , a pre -interpolation pixel row setup portion 
142, a pre-interpolation pixel row setup portion 143, a 
matching pixel-row operating portion 144 and a correcting 
means 145. 

Fig. 2 is a view showing a monitor screen on which a frame 
of visual image having a black fine line pattern on a white 
background is displayed as an example of an input video picture . 

In this drawing, 1 designates a monitor screen, 2 
designates an inclined fine line pattern and 3 designates 
an enlargement point in the input image shown in Fig. 3 as 
an object image to be interpolated in this embodiment. 

Fig. 3 is a chart showing input visual lines and lines 
to be interpolated at the enlargement point shown in Fig. 2. 

In this drawing. A, C, E, G and I designate input visual 
lines, and B, D, F, H designate lines to be interpolated. 
Designated at 21(A33 to A35) , 22(C27 to C31) , 23(E21 to E25) , 
24(G15 to G19) and 25(il0 to il3) , are black pattern portions 
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depicting inclined fine line pattern 2. The other area 
represents white pattern portions. 

As shown in the same figure, when a field of image is 
extracted, the pattern gaps become large at pattern edges 
and as the inclination of a fine line pattern becomes closer 
to the horizontal. Therefore, when a frame of image is 
generated by interpolation, the resultant image may present 
patterns with gaps and jagged outlines unless appropriate 
interpolation is performed. 

Next, the operation of the image interpolation system 
in accordance with the present embodiment will be described 
with reference to Figs.l to 8. 

First , generation of virtual interpolation line patterns 
by virtual interpolation line pattern generator 100 will be 
described. 

Input video picture input to inter-pixel differential 
luminance operating portion 101 of virtual interpolation line 
pattern generator 100 is assumed to have differential 
luminance values of ±127. In Fig. 3, the differential 
luminance value for every pair of opposing pixels on 
neighboring two input visual lines above and below, A and 
C lines , D and E lines , E and G lines and G and I lines , is 
calculated to obtain data of the pixels on lines to be 
interpolated. Then, in normalization processor 102, 
normalization process is performed based on a set threshold. 



and the data of the pixels on lines to be interpolated is 
classified into three classes , + , - and 0 . Pattern extracting 
portion 103 extracts segments of pixels continuously having 
the same class, either + or - , as patterns. 

For each of the extracted patterns, the average of the 
differential luminance values in the pattern is calculated 
by intra-pattern differential luminance operating portion 
111 of coring processor 110. Then, in pattern edit judging 
portion 112, each pattern is checked by comparing the 
aforementioned average value with pattern edit reference 
values such as a set threshold, minimum number of dots, maximum 
number of dots, etc. For example, the area having an average 
value not reaching the threshold is determined as an 
inconspicuous area or indistinct area and is excluded and 
deleted from the patterns by pattern edit implementing portion 
113. 

Pattern edit judging portion 112 may also perform edit 
judgement for pattern extension and connection between 
patterns close to each other by comparing their signs. 

Fig. 4 is a chart showing virtual interpolation line 
patterns generated through the above process by virtual 
interpolation line pattern generator 100. 

In this figure, 41 to 44 designate virtual interpolation 
line patterns classified as the class{-) , 51 to 54 designate 
virtual interpolation line patterns classified as the 
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class (+) and others are virtual interpolation line patterns 
classified as the class{0). 

In virtual interpolation line pattern generator 100, 
as shown in Fig. 4, for the virtual interpolation line pattern 
separated for each line to be interpolated, the re-calculated 
average of differential luminance values, the starting 
position, length, sign(+,-). the average differential 
luminance and other data of each pattern are created so that 
the data is sent to virtual interpolation database 120. In 
virtual interpolation database 120, the data thus obtained 
from the processing is stored as virtual interpolation pattern 
data. The amount of data being stored may be for the entire 
field or may be for a multiple number of lines required to 
be processed by interpolation segment determining processor 
130 and interpolation implementing portion 140. 

Next, the determination process of the segments to be 
interpolated and the direction of interpolation by 
interpolation segment determining processor 130 will be 
described. 

Fig - 5 is a chart for illustrating how the first matching 
pattern searching portion 131 searches for matching patterns. 

When the virtual interpolation data is input from virtual 
interpolation database 120 to first matching pattern search 
portion 131 of interpolation segment determining processor 
130, first matching pattern search portion 131 searches for 
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matching patterns as to each of virtual interpolation line 
patterns 41 to 44 and 51 to 54. For example, referring to 
a search for matching patterns of virtual interpolation line 
pattern 42, the segment of 21, 22, 23 and 24 on the virtual 
interpolation line D, based on the search range set up in 
search condition setup portion 133 and based on the matching 
judgment conditions such as the sign, segment length, average 
differential luminance value etc., set in matching pattern 
condition setup portion 132, virtual interpolation line 
patterns matching virtual interpolation line pattern 42 are 
searched for from the virtual interpolation line patterns 
located below, to the lower left and to the right on the line 
F , as indicated by the arrows shown in Fig . 5 . 

Next, if any patterns having the same sign, + or - and 
the same segment length, approximately average differential 
luminance were found in the search range as a result of the 
above search, matching pattern locating portion 134 , extracts 
the virtual interpolation line patterns as matching patterns 
with virtual interpolation line pattern 42 . Then , the pattern 
which has a center closest to the center of virtual 
interpolation line pattern 42 is searched for from the 
extracted matching patterns , one from each side of the central 
axis of virtual interpolation line pattern 42. As a result, 
virtual interpolation line pattern 43, the segment of 15, 
16, 17 and 18 on virtual interpolation line F, existing on 
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one side only, is identified as the matching pattern. 

Further, since in actual video pictures, there are cases 
where matching patterns exist in multiple directions or where 
fine lines having gaps and edges exist, a vector search is 
performed in order to avoid mal -extraction of matching 
patterns . 

Directional vector extracting portion 135 extracts 
vectors originating from the base pattern to the candidate 
patterns extracted as the matching patterns. In this case, 
a vector from the base pattern, i.e. , virtual interpolation 
line pattern 42, to candidate pattern 43 extracted as the 
matching pattern is extracted. Subsequently, in second 
matching pattern searching portion 136, analogous patterns 
that lie in the direction of the vector and meet the matching 
Judgment conditions are searched for based on the conditions 
set up in search condition setup portion 133 and matching 
pattern condition setup portion 132. In this way, if an 
analogous pattern is found in the vector direction based on 
the vector search in directional vector extracting portion 
135 and second matching pattern searching portion 136, the 
pattern is determined to be the segment to be interpolated 
and the candidate pattern which lie in the direction in which 
many number of analogous patterns exist is selected as the 
matching pattern. Here, if no analogous pattern is found in 
the vector direction, no interpolation is implemented for 
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that segment. Thus, it is possible to avoid mal- extract ion 
of a match pattern for a fine line and in the direction of 
an edge. 

Fig. 6 is a chart for illustrating how directional vector 
5 extracting portion 135 and second matching pattern searching 
portion 136 search for a vector. 

In this figure, a vector, indicated by the arrow, from 
the base pattern, i.e., virtual interpolation line pattern 
42 to candidate pattern 43 , is extracted by directional vector 

10 extracting portion 135. More matching patterns lying in the 
vector direction are found on the lines (lines B and H) by 
second matching pattern searching portion 136. As a result, 
virtual interpolation line pattern 41 of 27, 28, 29, 30 and 
31 on virtual interpolation line B and partially shown, virtual 

15 interpolation line pattern 44 of (9), 10, 11, 12 and 13 on 
line H, are found to be the matching patterns , whereby virtual 
interpolation line pattern 42 is determined to be the segment 
to be interpolated in the input pattern directed along the 
vector. At the same time, pattern 43 may be also determined 

20 as the segment to be interpolated as well as virtual 

interpolation line pattern 42. The designated segments to 
be interpolated, the direction for interpolation, etc. , 
determined by interpolation segment determining portion 130 
may also be registered into virtual interpolation database 

25 120. 
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As the determining method in interpolation segment 
determining portion 130, depending on the line on which the 
matching patterns exist in the vector direction or depending 
on the number of matching patterns existing on the same line 
directed along each vector with respect to the central axis 
of the base pattern, that is, depending on the distribution 
of matching patterns, search for match patterns may be 
implemented allotting different weights , or based on dedicated 
matching pattern determining conditions for the vector search 
mode . In the present embodiment , whether or not interpolation 
is needed and the direction of patterns to be interpolated 
are determined depending on the number of matching patterns 
existing in the vector direction, but the determining method 
should not be limit to this . 

Next, the interpolation implementing process in 
interpolation implementing portion 140 will be described. 

Pixel -row match searching portion 141 receives an input 
video picture and extracts pixel-row matches corresponding 
to the segments to be interpolated from the input visual lines 
above andbelow in the input field of picture . under the matching 
conditions, search range and other conditions, set by search 
condition setup portion 142, based on the 

interpolation- designated segments and the direction to be 
interpolated, input from interpolation segment determining 
processor 130. 
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Then, pre -interpolation pixel row setup portion 143, 
in order to determine the pixel rows to be written over the 
interpolation-designated segments, sets up 
pre-lnterpolation pixel rows, each of which at least includes 
5 part of its pixel -row match, based on the positional data 
of the pixel-row matches, the segment starting position and 
length of the matching pattern between the neighboring lines , 
obtained from virtual interpolation database 120. 

In the present embodiment, since the length of the 
10 interpolation-designated segment and that of the pixel-row 
match on the visual line are represented identically, the 
pre- interpolation pixel row corresponds to the pixel-rowmatch 
itself. 

In Fig. 7, 61 to 63 and 71 to 73 designate pre-interpolation 
15 pixel rows {=pixel- row matches) set up by pre-interpolation 
pixel row setting portion 143. 

Next, matching pixel-row operating portion 144 
calculates the average video data values of upper pixel -row 
matches 61 to 63 and lower pixel-row matches 71 to 73 and 
20 positional data, and based on the average and positional data 
of the video data thus calculated, interpolation implementing 
portion 140 implements interpolation by overwriting the 
patterns of the average video data on the lines to be 
interpolated, at the mid positions between upper pixel- row 
25 matches 61 to 63 and lower pixel-row matches 71 to 73. 
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Fig. 8 shows pixel rows 81 to 84 which are interpolated 
by Interpolation implementing portion 140. 

It should be noted that pixels other than the 
interpolation- designated segments on the lines to be 
interpolated are interpolated based on a prior art 
interpolation technique. Here, examples of the prior art 
interpolation technique include the method of interpolating 
the lines to be interpolated with average video data of the 
input lines above and below and the method of interpolating 
the lines to be interpolated with the video data of the input 
lines above - 

Next, the second embodiment of the present invention 
will be described with reference to Figs. 9 to 11. 

Fig. 9 is a block diagram showing a configuration of an 
image interpolation system in accordance with this embodiment . 

The configuration of the image interpolation system of 
this embodiment sets up pre- interpolation pixel rows based 
on a virtual interpolation database, so that the pixel-row 
match searching portion 141 and search condition setup portion 
142 needed in the interpolation implementing portion 140 of 
the first embodiment become unnecessary. 

That is, interpolation implementing portion 140 is 
comprised of a pre -interpolation pixel row setup portion 143, 
a matching pixel-row operating portion 144a and a correcting 
means 145. 
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Next, the operation of the image interpolation system 
In accordance with this embodiment will be described with 
reference to Figs . 9 to 11. 

To begin with, the positional relationship of pixel rows 
on the visual lines and on the lines to be interpolated is 
clearly shown in Figs. 10 and 11. 

In Fig. 10, visual pixel rows(black) are designated by 
21 to 25(A33 to A34, C27 to C31, E21 to E25 , G15 to G19 and 
no to 113); interpolation-designated segments(-) are 
designated by 41 to 44(B27 to B31, D21 to D25, F15 to F19 
and HIO to H13) , interpolation-designated segments{+) are 
designated by 51 to 54(B33 to 834, D27 to D31, F21 to F25 
andHlS tol9) ; and pre-interpolation pixel rows are designated 
by 61 to 64 and 71 to 74(A30 to A34, C24 to C28, C30 to C34, 
E18 to E22, E24 to E28, G12 to G16, GIB to G22 and 112 to 
116) . 

In Fig. 11, visual pixel rows(black) are designated by 
21 to 25(A33 to A34, C27 to C31, E21 to E25, G15 to G19 and 
no to 113); and interpolation pixel rows are designated by 
81 to 84(B27 to B31, B33 to B34, D21 to D25, D27 to D31, F15 
to F19, F21 to F15, HIO to H13 and H15 to H19). 

First, referring to an example of setting up a pixel 
row for interpolation in interpolation-designated segment 
42(D-21 to 25), pre-interpolation pixel row setup portion 
143, based on the segment starting positions and length data 
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Of interpolation -designated segments 42 and 43(D-21 to D25, 
F-15 to F19) obtained from virtual interpolation database 
120, sets up pre-interpolation pixel rows 62 and 63(C24 to 
C28, E18 to E22), on lines C and E. 
5 These are determined based on the starting positions 

and length data of interpolation-designated segments 42 and 
43(D21 to D25, F15 to F19) and are the segments which are 
enclosed by the straight lines which pass by the segment 
ends(D21 to F15. D25 to F19) intersecting each of lines C 
10 and D. 

Next, matching pixel-row operating portion 144 
calculates the average of the paired pixels of 
pre-interpolation pixel rows 62 and 63{C24 to C28 and E18 
to E22) and sets up an interpolation pixel row 82(D21 to D25)as 

15 shown in Fig. 11. 

Then, the average of pixel data of C24 and E18 of 
pre-interpolation pixel rows 62 and 63(C24 to C26 and E18 
to E22) is calculated to produce the pixel data of D21 of 
interpolation pixel row 82 {D21 to D25) . Thus, average of pixel 

20 data is calculated sequentially up to C28 and E22 so as to 
produce the pixel data of interpolation pixel row 82(D21 to 
D25) . 

By the above process, for each interpolation-designated 
segment an interpolation pixel row is produced, and these 
25 interpolation pixel rows are written over the pixels of the 
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Virtual interpolation patterns by the Interpolation pixel 
row while correcting means 145 adds corrections in harmony 
with the pixels around the lines to be interpolated, to thereby 
implement line interpolation. 
5 It should be noted that pixels other than the 

interpolation -designated segments on the lines to be 
interpolated are interpolated based on a prior art 
interpolation technique. Here, examples of the prior art 
interpolation technique include the method of interpolating 
10 the lines to be interpolated with average video data of the 
input lines above and below and the method of interpolating 
the lines to be interpolated with the video data of the input 
lines above . 

Data of pixels B32, D26, F20 and H14 in Figs. 10 and 11 
15 can be created by pattern edition in coring portion 110 or 
by correcting means 145. 

Next , the third embodiment of the present invention will 
be described with reference to Figs. 12 to 13. 

Fig. 12 is a block diagram showing a configuration of 
20 an image interpolation system in accordance with this 
embodiment . 

In this figure , 146 designates a coincident interpolating 
direction confirming portion. Pixel-row match searching 
portion 141 needed in interpolation implementing portion 140 
25 in the first embodiment is replaced by coincident 



interpolating direction confirming portion 146. 

Fig. 13 is a chart for illustrating positional 
relationship between the pixel rows on visual lines and the 
lines to be Interpolated. 
5 Next, the operation of the image interpolation system 

in accordance with this embodiment will be described with 
reference to Figs. 12 to 13. 

Coincident interpolating direction confirming portion 
146, based on the information obtained from virtual 

10 interpolation database 120 as to the direction in which the 
interpolation-designated segment should be interpolated, 
under the matching conditions, search range and other 
conditions, set by search condition setup portion 142, 
extracts an adjacent interpolation-designated segment on the 

15 same line as each interpolation-designated segment so as to 
perform error detection as to the interpolation-designated 
segment and extension of the segment to be interpolated. The 
result is given, as shown in Fig. 13, as extended 
interpolation-designated segments (-) 41 ' to 44'(B27 to B32, 

20 D21 to D26, F15 to F20 and HIO to H14), which are created 
by extending interpolation-designated segments{-) 41 to 
44(B27 to B31, D21 to D25, F15 to F19 and HIO to H13) shown 
in Fig. 10. 

The operation of the present embodiment downstream is 
25 the same as the second embodiment, so the description is 
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omitted. 

In the description of each of the above embodiments, 
luminance data is used as pixel data for inter- pixel operation 
in each process, but primary color data or chrominance data 
5 may be used instead. 

Each means in the above embodiments may be configured 
of hardware or the processing flow can of course be implemented 
by software processing. Also in this case, the same result 
can be obtained. 

10 In accordance with the inventions of the above 

embodiments, in contrast to the typical line interpolation 
performed on a pixel basis, line interpolation is performed 
with patterns which are obtained by performing arithmetic 
operations between the matching patterns which are extracted 

15 from the input visual lines, adjacent to the segments to be 
interpolated, it is possible to render almost horizontal 
inclined edges and fine lines in a smooth manner. 

Since operations between input visual lines and various 
types of coring processes are performed so as to make virtual 

20 interpolation patterns into database which can be referred 
to for detennination of segments to be interpolated and during 
implementing interpolation, exact interpolation can be 
implemented with mis judgment prevented, to thereby make the 
total operation processing efficient, thus enabling real-time 

25 interpolation. 
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Industrial Applicability 

As has been described, according to the inventions 
defined in the above first, second, twenty-ninth and thirtieth 
aspects , real-time interpolation is made possible by achieving 
5 sharp reduction in processing speed. Even when visual 

patterns appear with gaps at inter- lines in the field of image , 
interpolation is performed based on matching pixel rows on 
the adjacent lines, so as to reproduce a frame of image with 
smooth patterns without gaps . 

10 According to the inventions defined in the above third 

and thirty-first aspects , it is possible to search endportions 
of various image patterns including fine line patterns, 
pattern edges, etc. , in a short time and across the wide range 
and judge their conditions , thus making it possible to perform 

15 precise interpolation and depict smooth pattern edges. 

According to the invention defined in the above fourth 
aspect, based on the distribution of the extracted matching 
patterns, it is determined whether the associated segment 
should be interpolated or not and in which direction 

20 interpolation should be performed. Therefore, the 

conditions of fine line patterns and pattern edges can be 
judged exactly, thus making interpolation free from 
mal-detection possible. 

According to the invention defined in the above fifth 

25 aspect, based on the distribution of the extracted matching 
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patterns, it is determined whether the associated segment 
should be interpolated or not and in which direction 
interpolation should be performed. Therefore, even with a 
picture in which many analogous patterns exist , the conditions 
5 and positions of fine line patterns and pattern edges can 
he judged exactly, thus making interpolation free from 
mal-detection possible. 

According to the inventions defined in the above sixth 
to sixteenth aspects, since virtual interpolation data is 
10 constructed of pixel-rows as units of data, it is possible 
to smoo then pattern edges and perform operations in an improved 
efficiency. 

According to the inventions defined in seventeenth to 
twenty-eight aspects, since inconspicuous parts and 
15 indistinct parts which have little influence on the picture 
display can be erased or connected by coring, it is possible 
to sharply improve the speed of interpolation while 
maintaining image quality. 



20 
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CLAIMS 

1 . An image interpolation system for interpolating the gaps 
between the lines forming an image, comprising: 

a virtual interpolation data generating means for 
5 generating virtual interpolation data of inter- lines between 
the lines of the input image, based on the input image line 
data; and 

an interpolating means for interpolating the pixels 
between input image lines, based on the generated virtual 
10 interpolation data. 

2 . An image interpolation system for interpolating the gaps 
between the lines forming an image, comprising: 

a virtual interpolation data generating means for 
15 generating virtual interpolation data of inter- lines between 
the lines of the input image, based on the input image line 
data; and 

an interpolating means which, based on the generated 
virtual interpolation data, generates pre -interpolation 
20 pixels on the input image lines, and interpolates the pixels 
between input image lines above and below by performing mutual 
operations between pre -interpolation pixels generated on the 
input image lines above and below, 

25 3 . An image interpolation system for interpolating the gaps 
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between the lines forming an Image, comprising: 

a virtual interpolation data generating means for 
generating virtual interpolation data of inter-lines between 
the lines of the input image, based on the input image line 
5 data; 

an interpolation segment determining means for 
determining segments to be interpolated between the input 
image lines and the direction of interpolation, based on the 
generated virtual interpolation data; and 

10 an interpolating means which generates 

pre- Interpolation pixels on the input image lines, based on 
the generated virtual Interpolation data and the determined 
data of the segments to be interpolated, determined by the 
interpolation segment determining means, and interpolates 

15 the pixels between input image lines based on the generated 
pre-interpolation pixels. 

4 . The image interpolation system according to Claim 3 , 
wherein the interpolation segment determining means 

20 comprises: a search condition setup means for setting up a 
pattern search range ; a matching pattern condition setup means 
for setting up matching pattern conditions; and a first 
matching pattern searching means for searching for matching 
patterns based on the conditions designated by the search 

25 condition setup means and by the matching pattern condition 
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setup means . 

5. The image interpolation system according to Claim 3, 
wherein the interpolation segment determining means 

5 comprises: a search condition setup means for setting up a 
pattern search range ; a matching pattern condition setup means 
for setting up matching pattern conditions; a first matching 
pattern searching means for searching matching patterns based 
on the conditions designated by the search condition setup 

10 means and by the matching pattern condition setup means; a 
directional vector extracting means for extracting the 
direction of the vector of the detected matching patterns; 
and a second matching pattern searching means for searching 
for matching patterns existing in the extracted direction 

15 of the vector , based on the conditions designated by the search 
condition setup means and by the matching pattern condition 
setup means . 

6. The image interpolation system according to Claim 1, 
20 wherein the virtual interpolation data generated by the 

virtual interpolation data generating means is constructed 
of units of pixel-rows of data. 

7. The image interpolation system according to Claim 1. 
25 wherein the virtual interpolation data generated by the 
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virtual Interpolation data generating means is constructed 
of units of pixel-rows of data. 

8. The image interpolation system according to Claim 3, 
5 wherein the virtual interpolation data generated by the 
virtual interpolation data generating means is constructed 
of units of pixel-rows of data. 

rij 9- The image interpolation system according to Claim 4, 

p 10 wherein the virtual interpolation data generated by the 

h,n virtual interpolation data generating means is constructed 

of units of pixel-rows of data. 

O 

m 10- The image interpolation system according to Claim 5, 

JI 15 wherein the virtual interpolation data generated by the 

virtual interpolation data generating means is constructed 

of units of pixel-rows of data. 

11. The image interpolation system according to Claim 1, 
20 wherein the virtual interpolation data generating means 

comprises: an inter-pixel operating means for calculating 
the difference in pixel data between the pixels on the 
neighboring input image lines; and a normalizing means for 
classifying the pixels into multiple classes according to 
25 the calculated value of the difference in pixel data; and 
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a pattern extracting means for extracting rows of pixels 
normalized and. classified in an identical class as patterns. 

12. The image interpolation system according to Claim 2, 
wherein the virtual interpolation data generating means 
comprises: an inter-pixel operating means for calculating 
the difference in pixel data between the pixels on the 
neighboring input image lines; and a normalizing means for 
classifying the pixels into multiple classes according to 
the calculated value of the difference in pixel data; and 
a pattern extracting means for extracting rows of pixels 
normalized and classified in an identical class as patterns. 

13. The image interpolation system according to Claim 3, 
wherein the virtual interpolation data generating means 
comprises: an inter-pixel operating means for calculating 
the difference in pixel data between the pixels on the 
neighboring input image lines ; and a normalizing means for 
classifying the pixels into multiple classes according to 
the calculated value of the difference in pixel data; and 
a pattern extracting means for extracting rows of pixels 
normalized and classified in an identical class as patterns. 

14. The image interpolation system according to Claim 4, 
wherein the virtual interpolation data generating means 
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comprises: an inter-pixel operating means for calculating 
the difference in pixel data between the pixels on the 
neighboring input image lines; and a normalizing means for 
classifying the pixels into multiple classes according to 
the calculated value of the difference in pixel data; and 
a pattern extracting means for extracting rows of pixels 
normalized and classified in an identical class as patterns. 

15. The image interpolation system according to Claim 5, 
wherein the virtual interpolation data generating means 
comprises: an inter-pixel operating means for calculating 
the difference in pixel data between the pixels on the 
neighboring input image lines; and a normalizing means for 
classifying the pixels into multiple classes according to 
the calculated value of the difference in pixel data; and 
a pattern extracting means for extracting rows of pixels 
normalized and classified in an identical class as patterns. 

16. The image interpolation system according to Claims 6 
through 10 , wherein the virtual interpolation data generating 
means comprises: an inter-pixel operating means for 
calculating the difference in pixel data between the pixels 
on the neighboring input image lines; and a normalizing means 
for classifying the pixels into multiple classes according 
to the calculated value of the difference in pixel data; and 
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a pattern extracting means for extracting rows of pixels 
normalized and classified in an identical class as patterns . 

17. The image interpolation system according to Claim 1, 
5 wherein the virtual interpolation data generating means 

comprises: an inter-pixel operating means for calculating 
the difference in pixel data between the pixels on the 
neighboring input image lines; a noimalizing means for 
classifying the pixels into multiple classes according to 

10 the calculated value of the difference in pixel data; apattern 
extracting means for extracting rows of pixels normalized 
and classified in an identical class as patterns; and a coring 
means for Judging the patterns extracted on the same line 
to be interpolated based on the predetermined threshold and 

15 editing them. 

18. The image interpolation system according to Claim 2, 
wherein the virtual interpolation data generating means 
comprises : an inter-pixel operating means for calculating 

20 the difference in pixel data between the pixels on the 
neighboring input image lines; a normalizing means for 
classifying the pixels into multiple classes according to 
the calculated value of the difference in pixel data; a pattern 
extracting means for extracting rows of pixels normalized 

25 and classified in an identical class as patterns; and a coring 
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means for judging the patterns extracted on the same line 
to be interpolated based on the predetermined threshold and 
editing them. 

5 19. The image interpolation system according to Claim 3, 
wherein the virtual interpolation data generating means 
comprises : an inter-pixel operating means for calculating 
the difference in pixel data between the pixels on the 
neighboring input image lines; a normalizing means for 

10 classifying the pixels into multiple classes according to 
the calculated value of the difference in pixel data; a pattern 
extracting means for extracting rows of pixels normalized 
and classified in an identical class as patterns; and a coring 
means for judging the patterns extracted on the same line 

15 to be interpolated based on the predetermined threshold and 
editing them. 

20. The image interpolation system according to Claim 4, 
wherein the virtual interpolation data generating means 

20 comprises: an inter-pixel operating means for calculating 
the difference in pixel data between the pixels on the 
neighboring input image lines; a normalizing means for 
classifying the pixels into multiple classes according to 
the calculated value of the difference in pixel data; a pattern 

25 extracting means for extracting rows of pixels normalized 
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and classified In an Identical class as patterns; and a coring 
means for judging the patterns extracted on the same line 
to be interpolated based on the predetermined threshold and 
editing them. 

5 

21. The image interpolation system according to Claim 5, 
wherein the virtual interpolation data generating means 
comprises: an inter-pixel operating means for calculating 
the difference in pixel data between the pixels on the 

10 neighboring input image lines; a normalizing means for 

classifying the pixels into multiple classes according to 
the calculated value of the difference in pixel data ; a pattern 
extracting means for extracting rows of pixels normalized 
and classified in an identical class as patterns; and a coring 

15 means for judging the patterns extracted on the same line 
to be interpolated based on the predetermined threshold and 
editing them. 

22. The image interpolation system according to Claim 16, 
20 wherein the virtual interpolation data generating means 

comprises: an inter-pixel operating means for calculating 
the difference in pixel data between the pixels on the 
neighboring input image lines; a normalizing means for 
classifying the pixels into multiple classes according to 
25 the calculated value of the difference in pixel data; a pattern 
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extracting means for extracting rows of pixels normalized 
and classified in an identical class as patterns ; and a coring 
means for judging the patterns extracted on the same line 
to be interpolated based on the predeteirmined threshold and 
5 editing them. 

23. The image interpolation system according to Claim 17, 
wherein the coring means comprises : an intra-pattern pixel 
operating means for calculating the average of the pixel 

10 differential values of individual pixels in each pattern; 
and a pattern editing means for judging the average value 
based on the predetermined threshold and editing patterns. 

24. The image interpolation system according to Claim 18, 
15 wherein the coring means comprises: an intra-pattern pixel 

operating means for calculating the average of the pixel 
differential values of individual pixels in each pattern; 
and a pattern editing means for judging the average value 
based on the predetermined threshold and editing patterns. 

20 

25. The image interpolation system according to Claim 19, 
wherein the coring means comprises: an intra-pattern pixel 
operating means for calculating the average of the pixel 
differential values of individual pixels in each pattern; 

25 and a pattern editing means for judging the average value 
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based on the predetermined threshold and editing patterns . 

26. The image interpolation system according to Claim 20, 
wherein the coring means comprises : an intra-pattern pixel 
operating means for calculating the average of the pixel 
differential values of individual pixels in each pattern; 
and a pattern editing means for judging the average value 
based on the predetermined threshold and editing patterns . 

27. The image interpolation system according to Claim 21, 
wherein the coring means comprises: an intra-pattern pixel 
operating means for calculating the average of the pixel 
differential values of individual pixels in each pattern; 
and a pattern editing means for Judging the average value 
based on the predetermined threshold and editing patterns . 

28. The image interpolation system according to Claim 22, 
wherein the coring means comprises: an intra-pattern pixel 
operating means for calculating the average of the pixel 
differential values of individual pixels in each pattern; 
and a pattern editing means for judging the average value 
based on the predetermined threshold and editing patterns. 

29 . An image interpolation method for interpolating the gaps 
between the lines forming an image, comprising: 
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a step of generating virtual Interpolation data of 
inter -lines between the lines of the input image, based on 
the input image line data; and 

a step of interpolating the pixels between input Image 
5 lines, based on the generated virtual interpolation data. 

30 . An image interpolation method for interpolating the gaps 
between the lines forming an image, comprising: 

a step of generating virtual interpolation data of 
10 inter- lines between the lines of the input image, based on 
the input image line data; and 

a step of generating pre-interpolation pixels on the 
input image lines based on the generated virtual interpolation 
data and interpolating the pixels between input image lines 
15 above and below by performing mutual operations between 

pre-interpolation pixels generated on the input image lines 
above and below. 

31 . An image interpolation method for interpolating the gaps 
20 between the lines forming an image, comprising: 

a step of generating virtual interpolation data of 
inter-lines between the lines of the input image, based on 
the input image line data; 

a step of determining segments to be interpolated between 
25 the input image lines and/or the direction of interpolation. 
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based on the generated virtual interpolation data; and 

a step of generating pre- interpolation pixels on the 
input image lines, based on the generated virtual 
interpolation data and the determined data of the segments 
to be interpolated, and interpolating the pixels between input 
image lines based on the generated pre-interpolation pixels. 
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ABSTRACT 

An Image interpolation system for interpolating the gaps 
between the lines forming an image, includes: virtual 
interpolation data generating means 100 and 120 for generating 
5 virtual interpolation data of inter- lines between the lines 
of the input image, based on the input image line data; and 
interpolating means 130 and 140 for interpolating the pixels 
between input image lines, based on the generated virtual 
interpolation data. 
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